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Summary 

The stratospheric ozone layer protects life on Earth from the harmful effects of solar 
ultraviolet radiation. The ozone layer is currently in a fragile state because of depletion 
caused by man-made chemicals, especially chlorofluorocarbons. The state of the ozone 
layer is being monitored and evaluated by scientific experts around the world, in order to 
help policy makers assess the impacts of international protocols that control the 
production and release of ozone depleting chemicals. Scientists use a variety ozone 
measurements and models in order to form a comprehensive picture about the current 
state of the ozone layer, and to predict the future behavior (expected to be a recovery, as 
the abundance of the depleting chemicals decreases). Among the data sets used, those 
from satellite-borne instruments have the advantage of providing a wealth of information 
about the ozone distribution over most of the globe. Several instruments onboard 
American and international satellites make measurements of the properties of the 
atmosphere, from which atmospheric ozone amounts are estimated; long-term 
measurement programs enable monitoring of trends in ozone. However, the 
characteristics of satellite instruments change in time. For example, the instrument 
lenses through which measurements are made may deteriorate over time, or the satellite 
orbit may drift so that measurements over each location are made later and later in the 
day. These changes may increase the errors in the retrieved ozone amounts, and degrade 
the quality of estimated ozone amounts and of their variability. Our work focuses on 
combining the satellite ozone data with global models that capture atmospheric motion 
and ozone chemistry, using advanced statistical techniques: this is known as data 
assimilation. Our method provides a three-dimensional global ozone distribution that is 
consistent with both the satellite measurements and with our understanding of processes 
(described in the models) that control ozone distribution. Through the monitoring of 
statistical properties of the agreement between the data and the model, this approach also 
enables us to detect changes in the quality of ozone data retrieved from satellite-borne 
instrument measurements. This paper demonstrates that calculations of the changes in 
satellite data quality, and the impact these changes on the estimates of the global ozone 
distribution, can assist in maintaining the uniform quality of the satellite ozone data 
throughout the lifetime of these instruments, thus contributing to our understanding of 
long-term ozone change. 
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Abstract. Ozone observations from the Solar Backscatter UltraViolet/2 (SBUV/2) instruments 
and/or the Earth Probe Total Ozone Mapping Spectrometer (EP TOMS) have been assimilated 
in near-real time at NASA’s Data Assimilation Office (DAO) since January 2000. The ozone 
data assimilation system at the DAO was used as a tool for detecting and characterizing 
changes in the observation errors. The forecast model captures the geophysical variability. A 
change in the observed-minus-forecast (O-F) residuals, which are defined as differences 
between the incoming ozone observations and the collocated short-term model forecast, 
indicates a change in the assimilation system. If the model and the statistical analysis scheme 
are stable, then it points to a modification in instrument characteristics or a retrieval algorithm. 
However, sometimes a change in the ozone O-F residuals is caused by differences in the 
availability of the meteorological observations or modifications in the meteorological 
assimilation system whose winds are used to drive the ozone transport model. The O-F 
residuals are routinely produced and monitored in the assimilation process. Using examples 
from the NOAA-14 and NOAA-16 SBUV/2 instruments, and the EP TOMS, we demonstrate 
that the monitoring of time series of O-F residual statistics is an effective, sensitive, and 
robust method for identifying time-dependent changes in the observation-error characteristics 
of ozone. In addition, the data assimilation system was used to assist in the validation of 
updated calibration coefficients for the NOAA-14 SBUV/2 instrument. This assimilation- 
based monitoring work is being extended to ozone data from instruments on new satellites: 
Environmental satellite (Envisat), Earth Observing System (EOS) Aqua, and EOS Aura. 


1. Introduction 

Satellite borne instruments provide a wealth of data about the Earth’s atmosphere with a 

continuous and often near-global coverage. A few examples are the TIROS Operational 
Vertical Sounder (TOVS) (Smith et al. 1979), the Atmospheric Infrared Sounder (AIRS) 
(Aumann et al. 2003), and the Total Ozone Mapping Spectrometer (TOMS) (McPeters et al. 

1998). Satellite data are core input data sets with ever increasing importance for modem data 
assimilation systems. These systems are used to provide estimates and forecasts of the state of 
the atmosphere. Improvements in the availability, accuracy, and assimilation of satellite data 
are major factors contributing to the improvement in the accuracy of weather forecasts 
(Hollingsworth et al. 2002). Careful monitoring of the errors in the satellite data is critical for 
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their optimal usage within assimilation systems. Improvements in understanding of satellite 
data and their errors contribute to increased accuracy of weather and atmospheric constituent 
forecasts, potentially enable improvements in forecast models, and can ultimately lead to a 
better understanding of the underlying geophysical processes. 

Validation of retrieved data from satellite borne instrument measurements is a difficult task 
that can be complicated by many factors ( e.g . Fetzer et al. 2003). For newer instruments with 
high spectral and fine spatial resolution, such as AIRS, the large data volume makes the 
validation a very demanding task. Thus, development and use of sensitive, automated 
procedures that can aid in the monitoring and validation of retrieved satellite data is desirable. 

Traditionally, retrieved data products from satellite instruments have been validated in 
several ways. For example, algorithms used for their derivation are checked using synthetic 
and test input data (e.g. Froidevaux et al. 1996). Raw satellite-instrument radiances are 
monitored for abrupt changes and longer-term drifts. Zonal means of retrieved geophysical 
fields may be compared with the zonal means of model or assimilated fields (e.g. Gille et al. 
1996). The retrieved products are compared to data from the same instrument and from 
different instruments (e.g. Froidevaux et al. 1996, Remsberg et al. 2002). These comparisons 
are typically done using collocated data that are taken within certain distance and time from 
each other. The role of the collocation is to minimize the underlying temporal and spatial 
variability of geophysical fields. However, this dynamical variability is explicitly represented 
by the assimilated geophysical fields. This is one of the advantages of using assimilated fields 
in the validation of satellite data products. 

Data assimilation can contribute to validation of satellite data products. This approach 
capitalizes on the inherent ability of assimilation to handle large data volumes, often in near- 
real time, and facilitate a rapid feedback about the data quality to the instrument teams. Fields 
produced by assimilation into global models can provide a precise time and spatial collocation 
with every satellite measurement. The global nature of these fields allows application of 
averaging kernels, and production of a validation data set whose spatial representation is 
suitable for comparison with retrieved data. This is particularly important for instruments 
providing total or weighted columns of chemical constituents (e.g. Rodgers and Connor 2003). 

Data assimilation techniques have been applied to monitoring and validation of satellite- 
instrument observation errors. Piters et al. (1999) validated Global Ozone Monitoring 
Experiment (GOME) total column ozone observations using the Assimilation Model at the 
Royal Dutch Meteorological Institute (KNMI). They evaluated a random error in GOME and 
found a systematic error depending on the viewing direction. Similar approaches have also 
been successfully used in calibration and validation of other types of satellite observations. 
Stoffelen (1999) used wind fields from a numerical weather prediction model in calibration of 
the European Remote Sensing Satellite (ERS) scatterometers. This calibration is beneficial to 
the ERS geophysical product (the surface wind field over oceans). It is used for monitoring of 
the ERS scatterometers, and it can detect sudden instrument anomalies within six hours. Atlas 
et al. (1999) performed geophysical validation of NASA scatterometer (NSCAT) data sets. 
They compared NSCAT data with reference analyzed fields produced by assimilation of other 
data. When NSCAT data were assimilated, they found improvements in the resulting analysis 
and forecast fields. 
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In this paper we apply data assimilation techniques to monitoring of the error characteristics 
of satellite ozone data. In section 2 we give a brief overview of the data assimilation system 
and the approach to monitoring of observation errors that are used here. The subsequent 
sections contain examples from near-real time monitoring of errors in ozone data from three 
satellite-borne ozone instruments: the NOAA-14 Solar Backscatter Ultraviolet/2 (SBUV/2) 
instrument, the Earth Probe (EP) TOMS, and the NOAA-16 SBUV/2 instrument. In section 3 
we focus on the changes in the NOAA-14 SBUV/2 ozone profile data error characteristics, 
and their impact on the assimilated ozone fields. Monitoring of total column ozone from EP 
TOMS and NOAA-16 SBUV/2 instruments, and impacts of changes in the meteorological 
data assimilation system are covered in section 4. The use of global and regional statistics from 
the assimilation system is illustrated on several examples of algorithm and calibration 
modifications in the NOAA-16 SBUV/2 ozone profiles in section 5. Discussion of the results 
and plans for future work are given in section 6. 


2. Background 

Data assimilation (Daley 1991) provides a framework for combining available observational 
atmospheric data, along with their error characteristics, with a model prediction and its error 
characteristics to obtain the best estimate of the true atmospheric fields. This process yields 
analyzed or assimilated fields. In a sequential data assimilation process (see Fig. 1) the model 
transfers information from previous observational times to the current synoptic time. At the 
new synoptic time the model estimated field is updated with observations. In regions with 
dense observations the model estimate is, perhaps, changed substantially. In regions with 
sparse, or no, observations the estimate of the atmospheric field is largely determined by the 
model prediction. A statistical analysis scheme is used to produce an analysis field by 
combining the forecast field with the observations valid at this time according to their error 
characteristics. The cycle is then repeated. The forecast model can also be used to provide 
forecast fields for several days in advance. Data assimilation has been used successfully in 
numerical weather prediction, and increasingly also for obtaining distributions of atmospheric 
chemical constituents ( e.g . Fisher and Lary 1995, Long et al. 1996, Levelt et al. 1998, Eskes 
et al. 1999, Khattatov et al. 2000, Menard et al. 2000, Elbem and Schmidt 2001, Stajner et al. 
2001, Struthers et al. 2002). 

An integral part of the data assimilation process is the computation of differences between 
the observations and the collocated model predictions of these quantities 

(1) w °-HW, 

where w° is the vector of all the observations available at a single analysis time, H is the matrix 
of the (linear) observation operator, and w f is the vector of the model forecast on the analysis 
grid. These differences are the vectors of the observed-minus-forecast (O-F) residuals, and 
they allow us to evaluate how consistent the observations are with our understanding of the 
atmosphere as described by the forecast model. An O-F residual equals the difference between 
the observation error and the error in the forecast of the observed quantity. Thus, the statistics 
of O-F residuals provide information about the observation and forecast errors. Under the 
assumption that the forecast model does not change, and its error characteristics are constant in 
time, the differences in the time series of the O-F statistics are due to changes in the 
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observation error characteristics. An advantage in using the forecast as a standard against 
which observations are compared is that the forecast takes into account realistic geophysical 
variability of the fields. A disadvantage is that spatial and temporal variability of forecast 
errors complicates partitioning of the variability in O-Fs into the variability in forecast and 
observation errors, respectively. Implications for this work will be discussed below. 

This study uses near-real time O-F residuals produced by the operational ozone assimilation 
system at the NASA’s Data Assimilation Office (Riish0jgaard et al. 2000, Stajner et al. 2001). 
The assimilated ozone fields agree well with independent high quality ozone data. See Stajner 
et al. (2001) for some examples of the comparisons of assimilated ozone against WMO ozone 
sondes and Halogen Occultation Experiment (HALOE) (Briihl et al. 1996). In the initial 
configuration of the near-real time ozone assimilation system, the EP TOMS total column 
ozone (McPeters et al 1998) and the NOAA-14 SBUV/2 instrument ozone profiles (Bhartia 
et al. 1996) were assimilated into an off-line transport model between January 2000 and April 
2001. During April and May 2001 EP TOMS total column ozone and NOAA-16 SBUV/2 
profile observations were assimilated. Beginning in May 2001, only NOAA-16 SBUV/2 
ozone total column and profile observations were assimilated from May 2001. The off-line 
transport model (Lin and Rood 1996) used to predict ozone was driven by the assimilated wind 
fields from the version 3 of the Goddard Earth Observing System (GEOS) Data Assimilation 
System (DAS). Note that several operational changes were introduced into GEOS-3 during 
this period: their impacts will be discussed below. 

The DAO’s ozone assimilation system produces time series of O-F statistics. We routinely 
monitor these time series for abrupt changes. We use global and regional mean and root- 
mean-square (RMS) of O-F residuals for each data type. For profile observations these 
statistics are computed for each level. Another global statistic that we use is in the monitoring 
is x 2 , defined as follows. If the errors in w° and W are mutually uncorrelated Gaussian- 
distributed random vectors with mean 0 and covariances P and R, respectively, then the O-F 
residuals w°-Hw f are Gaussian distributed with the mean 0 and the covariance matrix 
HPIf+R. The random variable 

(2) z = (w°— //w* ) T (HPPF+R) - 1 (w°— tfw* ), 

then has the x 2 distribution with p degrees of freedom, where p is the number of observations 
(dimension of w°) at one analysis time. The mean of z is p and the variance is 2p. The mean of 
the normalized variable z/p is 1. Note that the representativeness error for TOMS total column 
data, whose footprint is smaller than the analysis model grid, is included as an additive 
component in the model used to evaluate R (Stajner et al. 2001). The SBUV footprint is 
comparable to the horizontal resolution of the analysis grid: thus the SBUV representativeness 
error need not be modeled explicitly. Initially, the x 2 was monitored to check how consistent 
the assumed error covariance models are with the realizations of the O-F residuals. However, 
we found that x 2 is a ls° convenient for detecting changes in the observation error 
characteristics (as will be illustrated in section 5). 

3. NOAA-14 SBUV/2 ozone profiles 

Ozone profiles retrieved from SBUV/2 instrument measurements are reported as partial 
ozone columns in 12 layers (Bhartia et al. 1996). Each of the layers 2 to 1 1 is approximately 5 
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km thick. Ozone partial columns from layers 3 (63-126 hPa) to 12 (0-0.25 hPa) are 
assimilated in our system (Stajner et al. 2001). In this section we focus on two examples of 
changes in NOAA-14 SBUV/2 error characteristics that can be seen from statistics of O-F 
residuals. A discussion of the impacts of the forecast model is then given. 

3.1 Features of the data 

The time series of the daily global mean of O-F residuals for NOAA-14 SBUV/2 in the 
layer 5, between 16 and 32 hPa, is shown by solid line in Fig. 2. Throughout the time series 
there is day-to-day variability. However, there is a sharp increase on March 31, 2000 (mark 
"A" in Fig. 2). The forecast model did not change in any way on that day, but the calibration 
of the SBUV/2 instrument was modified. After the SBUV instrument team reexamined the 
calibration coefficients, they derived updated coefficients and reprocessed the data (M. Deland 
and D. McNamara, personal communication). We used the reprocessed SBUV/2 data in a short 
assimilation run, and the change in the global mean O-F value on March 31 was reduced. 
These later calibration coefficients were operationally implemented by NOAA on August 8, 
2000 (M. Deland, personal communication), when the mean of O-F residuals slightly 
decreases (mark "V" in Fig. 2). 

The second feature that stands out in the time series of the mean of O-F residuals in Fig. 2 is 
the increase in November 2000 between marks "B" and ”C". This period coincides with the 
changes in the NOAA-14 SBUV/2 instrument grating positions. When a grating position 
changes, the wavelength at which the measurement is made differs from the nominal 
wavelength, and the actual wavelength is also reported with less accuracy. The wavelength 
shift to a less optimal region of the spectrum, together with a less accurate knowledge of the 
actual wavelengths at which measurements are made, can increase the errors in the ozone 
retrieval, from SBUV data. 

3.2 Sensitivity to forecast model 

Next, we examine the robustness of the temporal changes in the statistics of the O-F 
residuals to the use of different forecast models. The time series of the global mean O-F 
residuals for NOAA-14 SBUV/2 data in layer 5 from an ozone assimilation into a different 
ozone forecast model is shown by the dotted line in Fig. 2. This model is driven by winds from 
a prototype of the version 4 of GEOS meteorological assimilation, and it includes a chemistry 
scheme with parameterized ozone production and loss rates (Fleming et al 2001), where 
production rates were adjusted so that the quotient of production and loss rates in the upper 
stratosphere agrees with an ozone climatology based on SBUV data (Langematz 2000). In 
contrast, the operational assimilation driven by GEOS-3 fields (solid line in Fig. 2) does not 
include any chemistry, only the ozone transport, in the forecast model. Another difference 
between the operational GEOS-3 and prototype GEOS-4 meteorological systems is that the 
configuration of the operational system was changed a couple of times during year 2000, but a 
fixed version of the prototype GEOS-4 system was used for the entire year. The largest 
contribution to the mean differences between observations and forecasts comes from the 
tropical region. Both, upwelling and horizontal mixing reduce the ozone concentration in the 
SBUV/2 layer 5, just below the absolute mixing ratio peak near 10 hPa in the tropics. In the 
chemical parameterization, which is included in GEOS-4 driven ozone assimilation, there is 
net ozone production in this region. Thus, the mean differences between observations and 
forecast in SBUV/2 layer 5 are smaller in the GEOS-4 driven assimilation. However, despite 
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the offset between the O-F statistics from GEOS-3 and GEOS-4 driven assimilations, the 
features arising from the changes in the SBUV/2 data (near mark "A" and between marks "B" 
and "C") are robust, and can be seen in both GEOS-3 and GEOS-4 time series. 


3.3 Comparison of changes in O-F residuals and assimilated fields 

Statistics of O-F residuals can be very sensitive indicators of changes in observational error 
characteristics. Even though changes in zonal or global means of the observations can be 
relatively small, relative changes in O-F residuals can be easily noticeable. For example, the 
SBUV/2 calibration change on March 31, 2000 increased the zonal mean of ozone by about 1- 
2%, but the change in O-F residuals was about 10%. The latitudinal distribution of the impact 
of this SBUV/2 calibration change is seen in Fig. 3. The zonal RMS of SBUV O-F residuals 
increase throughout the tropics and in the northern middle latitudes. Most marked increase is 
seen near the equator and around 50°N to 60°N. There is almost no impact in the northern high 
latitudes or southern middle latitudes. 

The impact of the two different sets of SBUV/2 calibration coefficients on the assimilated 
field is seen in Fig. 4. Both analyses were compared with independent WMO ozone sonde data 
for the period from April 1 to April 10, 2000. There is a very small change in the mean 
profile. The RMS differences between sondes and the analysis with reprocessed SBUV/2 data 
are slightly smaller between 10 and 30 hPa, and slightly larger near 40 hPa, than for the 
analysis with the operational SBUV/2 ozone. The comparison against HALOE data was done 
using about 290 profiles in the Tropics and southern middle latitudes during the first 10 days 
in April 2000. This comparison also shows a very small impact from the change in the 
SBUV/2 data on the quality of the analyzed ozone. During the same period, we compared the 
assimilated ozone against POAM data (Lumpe et al. 2002) near 65°N and found a small 
impact, but a slightly improved mean profile shape with the reprocessed SBUV/2 data. The 
RMS differences for the analysis with reprocessed SBUV/2 data are smaller between 3 and 10 
hPa, and at 30 hPa, and larger elsewhere in the profile. Overall, we found that there was a very 
subtle impact on the assimilated ozone from the SBUV/2 calibration change on March 31, 
2000, which is much less pronounced than changes in O-F statistics. 

The impact of the grating position changes that occurred during November 2000 on the 
quality of the analyzed ozone field is shown in Fig. 5. The RMS difference between 
independent POAM profiles in southern high latitudes and the assimilated ozone is shown for 
November 13 to 30 (dashed line) and for December of 2000 (solid line). The comparison in 
November was done against 121 POAM profiles that cover between 70°S and 66°S. In 
December, 212 POAM profiles were used in the latitudes between 66°S and 63°S. All these 
southern latitudes that are covered by POAM are also observed by SBUV/2 data during 
November and December. A better agreement between POAM and assimilation in the middle 
stratosphere is seen during December, and this is consistent with the improvement in the mean 
of O-F residuals seen past mark "C" in Fig. 2. 

3.4 Discussion 

The above two examples of SBUV/2 calibration and instrument changes show that the 
statistics of O-F residuals from an assimilation system are indeed very sensitive to changes in 
the observations and their error characteristics. Furthermore, these statistics are collected in 
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near-real time, and allow a rapid feedback to the instrument team. They are also robust over 
systems driven by different meteorological data, and including or excluding the effects of the 
chemical processes. In contrast to the sensitivity of O-F residuals, the change in the 
assimilated ozone on March 31, 2000 was very subtle, and hard to detect from comparisons 
with independent high quality data (sondes, POAM, and HALOE). However, during the 
SBUV/2 instrument grating position changes in November 2000, both the agreement between 
assimilated ozone and POAM data and the mean of O-F residuals were notably degraded. 


4. Total ozone column data 

In this section we examine several properties of the total column ozone O-F residuals: their 
annual variability, their use for detection of cross-track biases of the scanning TOMS 
instrument, their changes due to a switch to another source of total column ozone data 
(NOAA-16 SBUV/2), and their sensitivity to changes in the meteorological assimilation 
system whose winds are used to drive the ozone assimilation model. 

4.1 EP TOMS total ozone column data 

An example of the time series of the global daily RMS of total column ozone O-F residuals 
is shown in Fig. 6. In 24 hours TOMS typically provides near-global coverage of the sunlit 
portion of the Earth. Abrupt short-term changes in Fig. 6 are typically caused by a reduced 
coverage of total ozone observations. For example, this occurs when total ozone data file is not 
available on time for the operational analysis. The only total column data that are available are 
then those from the end of the last orbit that started on the previous day because they are 
contained in the data file for the previous day. This happened on December 30, 2000 when we 
used only a part of one orbit going from southern high latitudes to the equator. In this region 
the ozone field has lower values and lower variability than in the northern middle latitudes. 
Therefore the RMS of O-F residuals for all observations on this day is about 9.2 DU, which is 
well below the usual range of RMS values in December 2000 of about 12 to 13 DU. The total 
ozone data were not available for almost 5 days. When the data returned on January 4 the O-F 
differences are larger (the RMS value is almost 15 DU) because of the accumulation of model 
errors over the 5 days for which total ozone column data were not available. The RMS of O-F 
residuals returns near 12 DU within one to two days after the TOMS data become available. 

A typical seasonal cycle is seen in Fig. 6 for the year 2000. The global RMS of total ozone 
O-F residuals is the lowest in January during the northern winter, when the northern high 
latitudes are in the polar night, unobserved by TOMS, and do not contribute to the global 
statistics. This RMS value peaks in the northern spring when the ozone values increase, and 
dynamical variability is large, especially in northern middle and high latitudes which are both 
observed by TOMS. The values then fall until July, and rise again until October. The 
variability corresponding to northern hemisphere winter and spring (January to April) is larger 
in magnitude than that corresponding to the southern hemisphere winter and spring. 

Starting in year 2000 the near-real time retrieved total column ozone from EP TOMS began 
developing a cross-track bias that grew over time and degraded the quality of the data. 
According to the TOMS processing team the bias appears to be due to a change in the optical 
properties of the front scan mirror of the instrument (news release on November 15, 2001 at 
http://toms.gsfc.nasa.gov/news/news.html') . This cross-track bias is illustrated by the scatter 
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plot in Fig. 7 (a). The TOMS O-F residuals at 2°S (where the geophysical variability in the 
total column ozone field is relatively small, except for the zonal wave number one) are shown 
by "x" marks for 14 orbits on January 28, 2001. The O-F residuals are plotted versus the 
model grid point across the orbit, where 1 denotes the westmost and 7 the esastmost grid point. 
The mean of O-F residuals for each of the grid points is shown by the line, and it increases 
from west to east by 13.1 Dobson units (DU). A year earlier, on January 28, 2000 the TOMS 
cross-track bias was smaller (Fig. 7 (b)) and the mean of O-F residuals for 14 orbits had a 
smaller variability of 7.6 DU. 

A departure from the typical seasonal cycle is visible from November of 2000 (just 
preceding mark "D" in Fig. 6). A strong upward trend begins and culminates with the peak in 
April 2001 (mark "E"). The increase is larger than that of a typical seasonal cycle. It was due 
to both the degradation of the quality of EP TOMS and a drift of the NOAA-14 orbit. The 
drift of the NOAA-14 orbit towards later afternoon Equator crossing caused a decrease of the 
SBUV/2 spatial coverage, an increase in the solar zenith angles of the measurements, and a 
consequent degradation of the ozone products (L. Flynn, personal communication). In April 
2001 (mark "E" in Fig. 6) we started assimilating NOAA-16 instead of NOAA-14 SBUV/2 
data. The RMS of O-F residuals started decreasing as the ozone profiles became better 
constrained by the NOAA-16 SBUV/2 observations with higher quality and better coverage 
than those from the NOAA-14 instrument. Since May 2001 (mark "F" in Fig. 6) total column 
ozone data from NOAA-16 SBUV/2 have been assimilated instead of those from EP TOMS. 
This slightly increased the day-to-day scatter of the RMS of the O-F residuals, but the RMS 
values in January 2002 returned within a range similar to that of January 2000. 

The time series of zonal RMS differences between total column ozone observations and the 
forecast is shown in Fig. 8. Lower values are typically found in the Tropics where both the 
total column ozone amounts and their spatio-temporal variability are lower than in 
extratropics. The RMS differences and the ozone amounts are lower only inside the Antarctic 
"ozone hole" which is seen in the southern high latitudes around October. The RMS 
differences are typically higher in middle latitudes, especially in winter and spring when total 
ozone amounts as well as their dynamical variability increase. Note that most of the buildup in 
the global RMS of total ozone O-F residuals (seen between marks "D" and "E" in Fig. 6) 
begins in northern middle latitudes in December 2000, extends to northern high latitudes in 
February 2001, and another buildup is seen in southern high latitudes around April 2001. 

4.2 NOAA-16 SBUV/2 total ozone column data 

During May 2001, when the ozone assimilation system started using NOAA-16 SBUV/2 
total column ozone data instead of EP TOMS data, an abrupt change occurs in spatial coverage 
(Fig. 8): the latitudinal coverage decreases by about 5° in the northern hemisphere, and by 
about 15° in the southern hemisphere. After several days of the initial adjustment of the 
system to the total column data from NOAA-16 SBUV/2, the RMS differences in the northern 
middle and high latitudes decrease. The RMS in the southern middle latitudes increases as 
expected from the previous annual cycle. The RMS values in the southern Tropics are just 
slightly higher than those seen in the year 2000 when the TOMS data were assimilated. 
However, the RMS values in the northern tropics between May and December of year 2001 
are higher than during previous periods. The SBUV/2 instrument provides measurements at 
nadir points only, for about 14 orbits per day. In contrast, EP TOMS is a scanning instrument 
that provides almost complete coverage of the Tropics every day. About 10,000 total column 
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observations were used daily from EP TOMS, compared to about 900 observations from 
NOAA-16 SBUV/2. With sparser total column ozone data from SBUV/2 the forecast relies 
more heavily on the model, and the O-F residuals are determined more strongly by forecast 
errors. 

4.3 Changes in the meteorological system 

The zonal RMS of O-F residuals (in Fig. 8) and the zonal mean of O-F residuals (not 
shown) decrease in December 2001 following several simultaneous changes in the 
meteorological GEOS-3 assimilation system. One of the changes in the meteorological system 
was use of in-house TOVS retrievals, including moisture data, instead of NESDIS TOVS 
retrievals. Another change was made to the radiation package of the general circulation model, 
to parameterize the effects of trace gasses. Assimilation of total precipitable water and the 
land-surface emissivities were also modified. The wind analysis increments were constrained 
so that the vertically averaged velocity potential vanishes. Forecast error variances were 
changed to be uniform on each level. However, all these changes to the meteorological system 
were introduced simultaneously, and it is not clear which ones had the largest effect on the 
ozone transport. 

4.3 Discussion 

We found that the total column ozone O-F residuals are sensitive to instrument cross-track 
biases. The size of O-F residuals changes with a switch in the instrument whose total column 
data are used. The O-F residuals are also sensitive to changes in the meteorological system 
used to produce winds that drive ozone transport in the ozone assimilation system. Thus, the 
ozone O-F residuals can be used for indirect evaluation of the transport properties of 
assimilated winds, especially in the stratosphere. 

5. NOAA-16 SBUV/2 ozone profiles 

Daily monitoring of the NOAA-16 SBUV/2 O-F residual regional statistics was 
implemented in May 2001. An example of the mean of the O-F residuals in the northern 
middle latitudes for the layer between approximately 4 and 8 hPa is shown in Fig. 9 (lower 
curve, labeled by "UL 7 mean"). Larger than typical variability can be seen in May, around 
October 9 and 13, and December 6. The change on October 13 was caused by the unintended 
use of NOAA-14 instead of NOAA-16 SBUV/2 data in the assimilation, and the bias between 
these two SBUV/2 data sets. Each of the other three jumps in the O-F statistics coincides with 
a change in either instrument calibration or in the operational retrieval algorithm used to 
produce NOAA-16 SBUV ozone data: the photomultiplier tube temperature correction was 
changed in the retrieval algorithm in May, the calibration was changed to use automatic inter- 
range ratio update using on-orbit data instead of extrapolated time dependent table on October 
9, and finally in December the calibration started using new time dependent albedo correction 
factors and the retrieval algorithm for pressure mixing of ozone estimates for different 
wavelength pairs was modified (S. Kondragunta, personal communication). 

One convenient statistic for monitoring of the changes in the O-F residuals is z/p (for z from 
eq. 2), the mean of x 2 statistics normalized by the number of observations. Recall that under 
ideal conditions its value should be one. This statistic accounts for all the O-F residuals 
according to their error covariances. It provides one global number representing all residuals 
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from different regions, levels, and observing conditions. In our experience, % 2 is indeed 
sensitive to changes in observation error characteristics for all the regions: it thus serves as an 
initial monitor of sudden changes, which prompt searches to determine the region where the 
changes occurred by inspecting mean and RMS statistics for different regions and levels. The 
time series of realizations of x 2 statistics from our system is shown in Fig. 9 by the upper curve 
(labeled by "% 2 "). For each of the large sudden changes in the mean of O-F residuals in the 
layer 4-8 hPa (Fig. 9, lower curve, labeled by "UL 7 mean") there is a corresponding sudden 
change in the time series of the % 2 statistics. 

Next, we compare the quality of assimilated ozone before and after the jump in % 2 in the 
beginning of December 2001. A comparison of assimilated ozone with HALOE data is shown 
in Fig. 10. The RMS differences between assimilation and HALOE data are shown for the 
second half of December 2001 (solid curves) and for November 2001 (dashed curves). The 
comparison is shown for all the regions (Tropics, northern and southern middle latitudes, and 
southern high latitudes) for which HALOE data are available. The SBUV/2 calibration and 
algorithm change on December 4 improved the agreement between the assimilated ozone and 
HALOE in all these regions. The largest improvement is seen in the upper stratosphere. These 
improvements are consistent with the improvement in the normalized mean of % 2 statistics seen 
from about December 10 in Fig. 9. 


6. Discussion and future work 

The monitoring of statistics of O-F residuals in an assimilation system was shown to be a 
sensitive procedure for detecting changes in the error characteristics of ozone observations 
from satellite-borne sensors. These monitoring results were demonstrated to be robust to use 
of different assimilating models, such as transport driven by GEOS-3 or GEOS-4 winds, and 
inclusion or exclusion of a parameterized chemistry scheme. We illustrated through several 
examples that the monitoring through assimilation is a potentially effective method for 
detecting changes in the errors of satellite ozone data. Time-dependent changes in the 
observation error characteristics of NOAA-14 SBUV/2, NOAA-16 SBUV/2, and EP TOMS 
instruments were detected. In the case of the NOAA-14 calibration change, the SBUV/2 
instrument team updated the calibration coefficients for the second time, we assisted in the 
validation of these updated calibration coefficients, and these coefficients were later 
implemented by NOAA in the operational processing of NOAA-14 SBUV/2 data. 
Comparisons of assimilated ozone fields with independent high-quality observations (from 
ozone sondes, HALOE, and POAM) indicate that the mean and RMS statistics of O-F 
residuals are very sensitive to changes in the input data quality. A noticeable change in O-F 
statistics corresponds to a subtle change in the quality of analyzed ozone fields in the case of 
NOAA-14 SBUV calibration modification in March of year 2000. In contrast, the increase of 
O-F residuals during November 2000 when NOAA-14 SBUV/2 experienced grating position 
anomalies is coincident with a clear degradation in the quality of assimilated ozone in the 
comparison with POAM data. Another case indicates the significance of % 2 statistic. The 
improvement in the values of nomalized mean of % 2 statistic during December 2001 was 
coincident with the improvement in the quality of assimilated NOAA-16 SBUV/2 ozone, 
which was evaluated through comparison with HALOE data. 
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Commonly used assumptions about unbiased observation and forecast errors in the 
assimilation are likely to cause partial propagation of the forecast and/or observation biases 
into assimilated fields. This limits the accuracy of the current operational assimilated ozone. 
Before interpreting changes in the ozone field as geophysically significant, their further 
validation is necessary. One approach is to compare the assimilated fields with independent 
observations. Inspection of the O-F statistics is also helpful. Our experience indicates that the 
most convenient statistics whose time series should be inspected is the mean of % 2 normalized 
by the number of observations. When changes are found we proceed to an inspection of 
regional statistic of O-F residuals. 

Environmental monitoring relies on the use of model assimilated observations from many 
data sources. A synthesis of all observational data with prediction models through statistical 
assimilation techniques, provides the only means possible of estimating consistent, 
multivariate fields of environmental parameters (NRC 1991). This synthesis is typically done 
in two ways: operationally in near-real time using different ever improving models, and in the 
"reanalysis" framework where a fixed state-of-the-art model and a fixed statistical 
assimilation technique are used over an extended historical period (Schubert et al. 1993, 
Kalnay et al. 1996, Gibson et al. 1997, Simmons and Gibson 2000). Both types of synthesis 
are affected by inevitable discontinuities between instruments or even the types of 
environmental observations that are available for usage. The validation statistics presented in 
this study contain some results applicable to reanalyses, while others are relevant only to 
"real-time" operational monitoring. Examples of the latter are changes in ozone forecasts 
brought about by the meteorological analyses (changes in GEOS-3 in December 2001) and by 
the combination of introducing a linearized ozone chemistry model at the same time as 
changing from GEOS-3 to GEOS-4 analyses (the "reanalysis" shown in Figure 2). The most 
important issues for reanalysis pertain to the unavoidable changes in instruments and to the 
time dependence of the data quality from any one instrument. Two important effects, the 
degradation of EP-TOMS data quality and the effective loss of NOAA-14 SMUV/2 data 
caused by the orbital drift, had marked impacts on the quality of the resultant ozone analyses, 
clearly characterized by the long-term behavior of the monitoring statistics. For reanalysis 
purposes, these effects must be considered. In a near-real time system, the re-calibration of 
SBUV/2 retrievals had an impact on analysis quality, an effect not immediately obvious from 
the analyses, but clearly evident in the O-F residuals; effects such as these, which can also be 
detected by careful monitoring of the retrieved data without assimilation, are crucial to the 
success and realism of analyses. The fundamental point about that analysis is not that 
assimilation can detect changes in the retrieval algorithms (these should be known anyway): it 
is that assimilation can isolate the impacts of such changes and, in the context of an end-to- 
end environmental monitoring system, can provide quantitative measures of these impacts and 
offer guidance into producing more appropriate changes that lead to a smaller shock to the 
system. The underlying message of this work is that careful use of the monitoring statistics, 
alongside the assimilated products, can yield a beneficial insight into the quality of the data 
and the suitability of any long-term analyses for inferring global change through quantitative, 
robust measures of the model-data uncertainty. 

Several instruments that measure ozone are included on the recent and planned satellites: 
Earth Observing System (EOS) Aqua, EOS Aura, and Environmental Satellite (Envisat). We 
have already applied the monitoring through assimilation for ozone observations from Envisat, 
from the AIRS instrument on EOS Aqua and from Microwave Limb Sounder onboard 
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NASA’s Upper Atmosphere Research Satellite. We plan to extend the monitoring through 
assimilation to retrieved ozone data from instruments on EOS Aura. 
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Figures 



Figure 1 . Schematic diagram of the ozone assimilation system with the major components and the data flow. 










1 «— 

Jan 1 



Jan 1 Marl Jun 1 Sep 1 Dec 1,2000 

Figure 2. Time series of daily global mean of NOAA-14 SBUV/2 O-F residuals in Umkehr layer 5 for 
operational SBUV/2 data (solid line) is shown for the system driven by GEOS-3 winds. The same quantity from 
the ozone assimilation driven by winds from a prototype GEOS-4 system, and including parameterized ozone 
chemistry is shown by the dotted line. Note the sharp jump near the mark "A" exceeding the typical day-to-day 
variability followed by a downward trend in the mean of O-F residuals. This feature is coincident with the 
instrument calibration change on March 31. Near the mark "A" the same quantity is shown for assimilation 
driven by GEOS-3 winds and using reprocessed SBUV/2 data (line with squares). An increase between 
marks"B” and ”C” coincides with a change in the grating position of the instrument. 
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Zonal RMS of SBUV 0-F residuals 
in Umkehr layer 5 (16-32 hPa) 
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Figure 3. Zonal RMS of the differences between SBUV/2 observations and forecast of ozone column (in 
Dobson units) between 16 and 32 hPa (Umkehr layer 5) is shown during the period of the sharp increase in the 
global mean differences near mark "A" in Fig. 2. The RMS differences increased most notably on March 31 in 
the Tropics and around 50°N to 60°N. 
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ozone mixing ratio [ppmv] 

Figure 4. Comparison of the analyses with operational SBUV/2 ozone (blue) and the reprocessed SBUV/2 
ozone (red) against independent WMO ozonesonde data (black) is given for April 1-10, 2000. The means of 
ozonesonde profiles and collocated analyses are shown by solid lines. The RMS differences between sondes and 
each of the analyses are shown by dashed lines. Umkehr layer 5 (16 to 32 hPa) is marked by the orange box. In 
this layer there is a small improvement in the agreement between sondes and assimilation when reprocessed 
SBUV/2 data are used. This is consistent with the lower mean of O-F residuals with reprocessed SBUV/2 data 
near the mark "A" in Fig 2. 
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Figure 5. Comparison of RMS differences between assimilated ozone mixing ratio and independent POAM 
ozone profiles in southern high latitudes is shown for November 13-30 and for December 2000. There is an 
improvement in the quality of assimilated ozone in the middle stratosphere in December, which is consistent 
with the decrease of the mean O-F residuals between November and December seen in Fig. 2. 
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Figure 6. Time series of daily global RMS of total column ozone O-F residuals. See text for details about 
marks "D"-"F". It shows a typical annual cycle during the year 2000. Following the mark "D" the RMS 
increases due to TOMS cross-track bias and the decrease in NOAA-14 SBUV/2 coverage. After the switching 
to the use of NOAA-16 SBUV/2 profiles (mark "E") and total columns (mark "F") the RMS decreases to near 
the levels seen in year 2000. 
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Figure 7. EP TOMS O-F residuals at 2° south latitude are shown by "x" marks as a function of the grid point 
across the orbit track. The westmost grid point is 1 and the eastmost is 7. The mean of O-F residuals for each 
grid point is shown by the line. The residuals and their mean are shown on January 28 of the year 2001 in (a) 
and year 2000 in (b). 
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Figure 8. Time series of the zonal RMS difference between total column observations and the forecast from the 
near-real time ozone assimilation system (in Dobson units) is shown. A typical distribution is seen in year 
2000: the lowest RMS differences are within the "ozone hole” region (in the high southern latitudes in 
September and October), relatively low differences are in the Tropics, and the highest differences are in middle 
to high latitudes during springtime both hemispheres. A buildup in the RMS differences is seen from about 
December 2000 to May 2001. The abmpt reduction in the latitudinal coverage of the total column ozone data in 
May 2001 occurs because the source of total column ozone data was changed from EP TOMS to NOAA-16 
SBUV/2. White areas indicate that data were not available for a specific time and latitude, e.g. within the polar 
night, or when a data file is missing for a day. 
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Figure 9. Time series of the daily mean of the global % 2 statistic normalized by the total number of observations 
is shown (upper curve, labeled "% 2 ") from May to December 2001. An example of a regional O-F statistic is 
shown by the lower curve (labeled by "UL 7 mean). It is the time series of the mean of O-F residuals (in 
Dobson units) in northern middle latitudes for the NOAA-16 SBUV/2 ozone in Umkehr layer 7 between about 
4 and 8 hPa. There is a close correspondence between the jumps in the two time series which exceed the typical 
day-to-day variability, e.g. in May, October and December. 
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Figure 10. Regional RMS differences between analyzed ozone and HALOE data before (dashed) and after 
(solid) the calibration and algorithm change in NOAA-16 SBUV/2 retrievals on December 4, 2001 are shown. 
Following the change in the retrievals there is generally an improvement in the quality of the assimilated ozone 
profile, especially at pressures less than 10 hPa. 




